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I. INTRODUCTION 

The combinatorial definition of entropy of Boltzmann 
and Planck 0] stands as one of the most profound 
equations of human discovery. Recently, it was used 
to derive the exact forms of the degenerate Maxwell- 
Boltzmann (MB), Bose-Einstein (BE) and Fermi-Dirac 
(FD) entropy functions, without the Stirling approxima- 
tion (i.e. the total number of entities, N, and/or the 
number of entities rij in each state i, and/or the degen- 
eracy do not necessarily approach infinity) 0- The ex- 
act entropy functions are supersets of their corresponding 
Stirling-approximate forms 0, IE IE 01 j but are distinct 
from other supersets such as the Tsallis entropy |8| • The 
analysis shows that the concept of information not only 
encompasses knowledge of the realization (complexion) 
of a system, but also knowledge of N, and for BE and 
FD systems, knowledge of the degeneracy of each state. 

The energy cost (in bits) of a "binary decision" - i.e. 
the cost of learning that a system, distributed over two 
equiprobable states, is in one such state - was calculated 
in each statistic. It was found that a binary decision 
costs < 1 bit in MB and BE systems, and in a narrow 
range within FD systems, if one has additional knowledge 
In BE systems, a zero cost is theoretically attainable. 
However, the cost is > 1 bit in BE and FD systems in the 
absence of such knowledge, except in the Stirling limits. 
In consequence, the observation of an individual BE or 
FD entity is thermodynamically irreversible (requires the 
input of energy or information); this explains the need to 
destroy a boson or fermion in order to observe it, and 
hence "the collapse of the wavefunction" Q . 

Exact MB, BE and FD statistics are further examined 



here. Firstly, an apparent paradox relating to quanti- 
sation effects is resolved. The previous analysis is then 
extended to consider an "s-fold decision" , i.e. the cost 
of learning that a system, distributed over s equiprob- 
able states, is in one such state. The findings gener- 
alise those reported previously 0. As before, the discus- 
sion draws irpon the combinatorial definition of entropy 
|jj lJ IE IE 1 the equivalence of information, energy 
and negative entropy |9L IXOL RLll Il2| : and the information- 
theoretic understanding of the second law of thermody- 
namics |E IH Gi- 



ll. BACKGROUND 

The Boltzmann pj - Planck combinatorial definition 
of entropy can be expressed as: 
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where H is a dimensionless entropy per unit entity; W is 
the statistical weight or number of possible realizations 
(complexions) of the system, of equal probability; N is 
the number of entities (e.g., atoms or molecules); and C 
is an arbitrary constant (reference datum). The weights 
for degenerate MB, BE and FD statistics are 
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where i denotes each distinguishable state, from a total 
of s distinguishable states; rii is the number of entities in 
each state i; gi is the degeneracy (multiplicity) of each 
state i; pi = rii/N is the probability of an entity being 
in state z; and on = g%/N is the relative degeneracy of 
state i. As noted it is mathematically convenient to 
consider the MB weight as a function of gi, and the BE 
and FD weights as functions of a, . 

From Eqs. (|1I4|1 . the exact entropy functions are 0): 

H X MB = ^E{-ln[feA)!] +PMm +PiNln 9i } 
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where superscript x implies the exact entropy. Here we 
take C = in each case, and also bring the external A! of 
the MB weight (Eq. |2J inside the sum using the natural 
constraint X)i=iPi = !• 

Applying the Stirling approximation to each factorial 
(In x\ — > x In x — x as x — > oo^, Eo^s. J|5I7|) reduce to their 



Stirling-approximate forms U, |5|, la, 13 : 
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from which the data are collected without regard to or- 
der. For s — 2 this reduces to the simple case of a 
coin tossed A times. In the Stirling limits A — » oo and 
rii — > oo, Vi, if the die is true (difficult to manufacture in 
practice), we have no difficulty with the statement that 
Pi = s _1 ,Vi; or for our coin, p\ = p2 = \. Alternatively, 
if a trickster can roll a die or toss a coin so that it always 
lands with face j up, we assign p i = \ for i = j and 
Pi = otherwise. 

What about the other extreme, at A = 1? Consider 
two situations: 

(a) If we have already rolled the die, and completely know 
the outcome, we can assign m = l,i = j;rii = 0,i ^ j 
for some j, hence pi = l,i = j;pi = 0, i j. Thus in a 
situation of complete knowledge, the numbers of entities 
in each state - and hence the probabilities - are quan- 
tised. For A > 1, this quantisation will generate discrete 
fractional values of pi. 

(b) On the other hand, if we do not know the outcome 
(e.g. before we toss the die), but consider that the die is 
true, we have no choice except to make the probability as- 
signments pi = s _1 ,Vi, using the principle of insufficient 
reason 01 > irrespective of the fact that the outcome will 
be quantised. If we convert these into numbers of entities 
within each state, we obtain nj = s _1 ,Vi. Similarly, for 
A > 1 we obtain rij = N/s, Vi. In other words, when 
we have no knowledge of the realization, the probability 
and entity assignments override the quantisation inher- 
ent in the problem. This does not mean that we have 
a simultaneous superposition of all faces uppermost on 
our die, nor a superposition of heads and tails on our 
coin (nor indeed, a superpositional alive-dead cat |14|): 
it only means that our description is based on an imper- 
fect state of knowledge. There is no need to demand that 
this description be physically realizable. 

Thus at low A, the analysis relies upon the Laplace- 
Jaynes interpretation of probabilities as "plausibilities" , 
or assignments based on what is known, rather than as 
measurable frequencies 0, . The author recognises 
that this view causes great difficulty for many physicists, 
but it is the only rational method with which to assign 
probabilities. Readers who do not understand this ap- 
proach are urged to thoroughly read Jaynes. Of course, 
in the Stirling limits A — > oo and rii — > oo, Vi, both the 
"before" and "after" scenarios can be physically realized, 
and can thus be conceptualised as measurable frequen- 
cies. 



IV. EFFECT OF AN s-FOLD DECISION 



III. AN APPARENT PARADOX 

Before proceeding, it is worth drawing out an appar- 
ent paradox in the implementation of "exact statistical 
mechanics" - not spelt out previously Q - which in fact 
is not a paradox. Consider a non-degenerate (i.e. <?j = 1) 
MB system, for example a die with s faces rolled A times, 



Now that the apparent paradox is understood, con- 
sider an "s-fold decision" as defined earlier, which en- 
compasses the above die problem. Before the decision, 
for equiprobable states we assign pi = s _1 , Vi. After the 
decision, we find that the system occupies only one state, 
whence pi = l,i = j;pi = 0, i j. Certainly, with in- 
creasing iV and s we will be more and more surprised 
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by this outcome, hence the information gained will also 
increase. The energy cost of the decision is @, 0, El : 

AE = -AI = Hfi na t - H inU ("nats") 

= S final ~ Sinit ("s-bits") (11) 

where AI is the information gain, limit and H final are 
the initial and final entropies (from Eqs. I|5I10|I ). and 
5 = H/ In s is a rescaled entropy. The units of "nats" 
reflect entropies based on the natural logarithm; however, 
it is more convenient to use "logical" logarithms of base 2 
for binary decisions ("bits"), base 3 for ternary decisions, 
etc; whence base s for s-fold decisions ("s-bits"). These 
are adopted in the last form of Eq. ^] using log s x = 
In x/ Ins. 

For finite N, provided we know N, we can use the exact 
form of Sinit in Eq. Ijll|l . However, if we don't know 
N, we must (by default) adopt the Stirling-approximate 
form of Si n u as the reference datum, since it reflects a 
state of higher entropy (lower information) . This leads to 
two separate costs: (a) the cost of learning N, given by 
AE N = -{Sf nit - S m \ t ); and (b) the cost of learning the 
realization, given by AE X = —(SJ inal — Sf nit ). The sum 
AE T r AE N + AE- = -{S x fmal - S?* u ) gives the total 
cost, in s-bits, of learning both N and the realization. 

By definition, an s-fold decision provides 1 s-bit of in- 
formation about some observable system. The cost AE 
must however be "paid for" from some external reservoir 
of negative entropy or information 0, ^| . Thus in 
calculating AE X or AE T , we are particularly interested 
in processes for which AE < 1 or > 1 s-bit, since these 
respectively produce a net information surplus or deficit 
in the combined observable-reservoir system, in apparent 
defiance of the second law of thermodynamics 
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CALCULATIONS 



The initial and final entropies and energy cost for 
an s-fold decision in each statistic, for both Stirling- 
approximate and exact cases, are listed in Table ]R(AE N 
and AE T are not listed but readily calculated). For 
convenience, it is assumed that gi = g,oti — a,Vi, and 
x\ = T(x + 1) for non-integer x. The resulting entropy 
surfaces - or the energy costs - can be plotted against 
their governing variables, to reveal their effect. In the 
following, although the plots are drawn as continuous in 
N and s, it is understood that both are discrete; further- 
more, all plots commence at the physically meaningful 
N = 1 and s = 2. 

For a non-degenerate MB system (g = 1), the refer- 
ence, initial and final entropies are plotted (in a negative 
sense) against N and s in Figure Q As shown, in this 
statistic the total cost is always 1 s-bit. In the Stirling 
limit N — > oo, AE N — ► 0, but the value of N above 
which the Stirling approximation can be considered valid 
increases with increasing s. At finite N, if one has knowl- 
edge of N, the decision can be completed for AE X < 1 
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FIG. 1: Information-energy diagram for an exact MB system 
for g — 1 



s-bit. Furthermore, this cost diminishes with increasing 
s (in the limit s — > oo, AE X — > 0). 

A plot of the various entropy functions for a degenerate 
MB system with g = 1000 (Figure^) reveals the effect of 
degeneracy. However, if we plot the energy costs (Figure 
I2t>), w e obtain surfaces which are identical to those in 
Figure 2] In other words, for exact MB statistics, the 
magnitude of the degeneracy has no effect on the net 
cost of an s-fold decision, regardless of N or s. 

The energy costs for an exact BE or FD system at 
a = 1000 (which very closely approximates a — > oo 0) 
are plotted in FigurelSJa, (the actual entropies, not shown, 
exhibit "folded" surfaces somewhat similar to Figure 0j). 
As evident, for N — > oo the cost of an s-fold decision is 
1 s-bit, but once again the N above which the Stirling 
approximation is valid increases with increasing s. How- 
ever, in contrast to the MB statistic, for finite TV the cost 
surfaces curve upwards, such that AE N increases signif- 
icantly as N — » 1, even as AE X decreases. The net effect 
is that with knowledge of N ^ oo, an s-fold decision can 
be obtained for AE X < 1 s-bit; however, without this 
knowledge, the total cost is AE T > 1 s-bit, except in the 
Stirling limit (N — > oo). 

In the limit as a — * oo, the various initial and final 
BE and FD entropies all become infinite; however, their 
differences approach the following values: 
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Plots of the latter two functions are essentially identical 
to those in Figure 
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FIG. 3: Information-energy diagrams of energy costs in exact BE and FD systems, for (a) a — 1000 (~ a — > oo); (b) exact BE 
system, a = 1; (c) exact BE system, a = j^; and (d) exact FD system, a = 1. 
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TABLE I: Initial and final entropies and energy cost of an s-fold decision, for Stirling-approximate and exact statistics 
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The effect of degeneracy on the BE statistic is illus- 
trated in FiguresOb-c, respectively for a = 1 and a = j^. 
In either case, if both N and a ^ oo are known, it pos- 
sible to complete the s-fold decision for AE X < 1 s-bit, 
even in the Stirling limit (iV — > oo). Indeed, a zero cost 
is theoretically attainable in either system at N = 1/a, 
independent of s (for N < I /a, the cost surfaces cross to 
produce AE X < 0; however, this occurs at a fractional 
degeneracy g < 1, and is thus non-physical). If only 
a =/= oo is known, the total cost is higher, but there exists 
a range of N and s for which AE T < 1. 

For the FD statistic, the effect of degeneracy is shown 
in Figure for a — 1 . As found previously 0] , in FD 
statistics, knowledge of a increases the cost of an s-fold 
decision (for N — > oo and s = 2, the cost is exactly 2 
bits). This effect does however diminish with increasing 
s. FD systems are therefore much less favourable to the 
generation of an information surplus, although there ex- 
ists a range of N and s for which AE X < 1 or AE T < 1 . 



not only with knowledge of its realization, but also with 
knowledge of N, and in the case of BE and FD statistics, 
with knowledge of a. MB systems are again found to be 
"well behaved" , in that the total cost is always equal to 1 
s-bit, regardless of N and s, whereas BE and FD do not 
satisfy this condition. With knowledge of N and/or the 
degeneracy, it is possible to achieve an s-fold decision for 
AE X < 1 s-bit in all systems examined (for FD systems, 
with some restrictions on N and s). Without knowledge 
of N or degeneracy, the total cost is always AE T > 1 
s-bit in BE and FD systems, but approaches 1 s-bit in 
the Stirling limit N — > oo. The findings generalise those 
reported previously 
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VI. CONCLUSIONS 

The above analysis extends the previous finding Q 
that "information" about a physical system is connected 
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